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Research  performed  under  this  grant  centered  on  the  computations  done  by  biological 
networks  of  nerve  cells,  and  on  the  theoretical  and  engineering  underpinning  of  neurobiological 
computation.  The  biological  studies  were  in  greatest  part  theoretical  analysis  of  the  olfactory 
(smell)  system.  This  is  a  very  old  and  "simple"  sensory  system,  and  is  highly  conserved. 
Animals  as  different  as  snails  and  humans  have  strong  a-nalomioal  t,iiiiilarities  in  their  olfactory 
systems.  In  addition,  the  basic  architecture  of  much  of  old  cortex  is  claimed  to  be  related  to  that 
of  olfactory  cortex,  so  understandings  gained  in  the  olfactory  system  are  potentially  applicable 
to  other  areas  of  brain. 

During  this  grant  period,  our  initial  work  was  in  extending  earlier  modeling  of  the 
olfactory  system  of  the  garden  slug  Limax  Maximus  to  include  the  kind  of  preprocessing  which 
would  be  essential  to  a  categorical  memory  for  odors  based  on  the  relative  strengths  of  a 
multiplicity  of  components.  [6].  We  then  turned  to  modeling  the  olfactory  bulb,  the  first  stage 
of  olfactory  information  processing  in  vertebrates.  The  explicit  intent  was  two-fold:  to  see 
whether  a  simplified  version  of  the  biophysics  of  the  olfactory  bulb  could  capture  the  essential 
aspects  of  the  oscillatory  behavior  seen  in  electrophysiological  recordings  in  bulb;  and  to  try  to 
understand  the  computational  function  of  the  bulb.  In  the  first  regard,  great  simplification  does 
appear  adequate,  and  this  opens  the  way  to  analytical  mathematical  approaches  [11].  The 
attempt  to  define  ih£  computational  role  of  the  oscillations  was  less  successful  [11,  12].  While 
2  consistent  view  of  the  role  of  oscillations  was  constructed,  it  as  yet  lacks  a  compelling 
rationale  in  neurobiology.  So  the  question  of  whether  the  view  put  forward  is  Ihg  correct  view 
of  the  oscillations  must  await  better  experiments  designed  to  test  our  theoretical  ideas.  Because 
of  the  success  of  the  simplified  analytical  model  of  bulb,  we  also  initiated  an  analysis  of  a 
simplified  model  appropriate  to  primary  olfactory  cortex  (prepiriform  cortex).  As  in  bulb, 
the  idea  is  to  validate  (or  invalidate)  such  a  model  on  the  basis  of  comparisons  of  simulations  of 
a  detailed  model,  real  electrophysiology,  and  simulations  of  the  detailed  model.  A  success  in  this 
direction  would  give  confidence  to  a  tractable  mathematical  description  of  cortical  function. 
However,  this  work  could  only  be  initiated,  and  not  completed. 

A  second  major  focus  of  research  centered  on  understanding  network  computation  at  a 
more  abstract  level.  Any  real  computing  system,  manmade  or  biological,  is  a  physical  system 
which  computes  by  having  a  "state"  which  changes  in  time.  Computation  is  fundamentally  a 
dynamical  process,  and  both  digital  and  analog  computation  can  be  so  represented.  Many  of  the 
papers  written  emphasize  this  importance  of  understanding  dynamics,  particularly  papers  [1], 
[3],  [4],  and  [15].  Others  exploit  the  dynamics  in  order  to  solve  some  particular  problem. 

For  example,  [18]  uses  the  dynamics  of  synapse  change  during  development  to  understand 
patterns  of  cell  response  in  early  visual  processing.  Papers  [7]  and  [10]  exploit  available 
dynamics  to  organize  time  delays  in  such  a  way  that  short  stereotype  signals  occurring  over 
time  (such  as  a  spoken  word,  a  bird  call,  or  a  facial  gesture)  can  be  recognized  as  a  whole. 
Conserved  quantities,  or  quantities  whose  change  is  monotonic  in  time  ("energy  functions") 
have  been  very  useful,  both  in  understanding  the  dynamics  of  neural  networks  and  in 
engineering  networks  to  perform  particular  tasks.  We  have  noted  [15]  that  the  kind  of  network 
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found  in  ft  e  olfactory  bulb  is  a  “Lagragian  network."  This  fact  represents  a  new 
“programming  tool"  for  understanding  more  complex  network  computation. 

Interest  in  applying  ideas  from  biological  neural  networks  to  real  problems  of 
engineering  raises  the  issues  of  how  best  to  implement  artificial  neural  networks  in  VLSI.  In 
particular,  the  same  network  and  algorithm  can  be  embedded  in  VLSI  in  two  separate  ways, 
either  a  digital  form  or  in  an  analog  form.  An  analysis  of  this  situation  [14]  indicates  that  a 
digital  implementation  is  always  at  least  as  fast  as  the  analog  version  and  that  the  analog  version 
is  much  less  complex.  The  compromises  and  important  considerations  for  the  tradeoffs  between 
these  include  questions  such  as  (1)  Can  the  analog  device  physics  itself  be  usefully  exploited  by 
the  algorithm?  (2)  Is  excess  accuracy  available  in  the  fabrication  process?  (3)  Is  absolute 
speed  truly  a  valid  criterion?  (4)  Is  power  availability  limited?  Tne  most  signiiiccuii  poini  is 
the  question  of  the  usefulness  of  the  device  physics  in  the  desired  algorithms.  Particularly  for 
sensory  processing,  logarithms,  exponentials,  time  delays,  and  frequency  band  filters  are 
extremely  useful  components  to  processing,  and  it  is  in  the  realm  of  “early  sensory 
processing"  that  fully  analog  neural  net  hardwares  appear  to  have  the  greatest  potential 
advantages. 

Biology  “computes"  on  the  molecular  scale  as  well  as  at  the  cellular  network  level 
represented  by  neurobiology.  The  translation  of  RNA  into  protein  sequence  is  readily  described 
as  an  algorithm  carried  out  by  computational  machinery  at  the  molecular  level.  The  folding  of  a 
protein  into  its  most  stable  three-dimensional  structure  is  the  solution  to  a  very  complex 
C'ptimization  problem.  Having  always  been  intrigued  by  these  facts,  we  have  spent  a  little  time 
exploring  them  for  what  they  can  tell  us  about  doing  computation  at  the  molecular  level.  We 
designed  a  shift  register  [5,  8,  9]  based  on  ideas  from  the  molecular  electronic  complex  used  in 
the  primary  processes  of  photosynthesis.  This  design  effort  was  directed  at  identifying  and 
solving  on  a  conceptual  level  some  of  the  difficult  problems  for  people  talking  about  "biochips" 
or  molecule-based  electronics. 

An  ability  to  readily  compute  the  three-dimentional  structure  of  a  protein  from  its 
primary  sequence  woula  be  a  major  boon  to  molecular  biology,  biochemistry,  and  biotechnology. 
This  problem  interacts  with  neural  network  theory  on  two  levels.  First,  it  provides  a  physical 
example  of  a  dynamical  system  solving  a  complex  problem  based  on  analog  variables.  Second, 
successfully  using  a  neural  net  on  a  problem  as  complex  as  this  in  a  non-trivial  fashion  will 
represent  a  large  step  forward  in  conceptualizing  neural  net  computation.  Our  very  small  first 
step  is  described  in  [19]. 
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